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ABSTRACT 

There  are  many  issues  associated  with  the  role  of  humans  in  automated  systems.  Among  those  that  have 
received  the  least  attention  are  the  questions  associated  with  uncertainty.  Uncertainty  is  inherent  in  military 
operations,  yet  it  is  often  ignored.  By  making  assumptions,  explicitly  or  implicitly,  about  the  environment  in 
which  it  works,  automation  can  modify  evidence  of  uncertainty,  conceal  the  sources  of  variability,  or  even 
induce  uncertainty  in  the  user  by  neglecting  environmental  variability.  This  paper  will  explore  the  range  of 
issues  and  suggest  some  ways  to  mitigate  the  effects  of  uncertainty  associated  with  automation. 


ASSUMPTIONS 

To  begin,  let  us  consider  the  assumptions  that  underlie  this  paper.  The  reader  may  not  agree  with  them. 
However,  they  are  the  firmly-held  opinions  of  the  author  and,  as  such,  must  be  made  explicit  for  the  reader  to 
comprehend  the  argument  herein. 

The  first,  and  most  important  assumption  is  that  automation  is  intended  to  assist  the  human,  not  the  other  way 
around.  It  is  the  human  who  is  ultimately  responsible,  and  therefore,  the  human  who  must  be  in  control. 
The  automation  and  the  human  must  work  together  to  accomplish  the  human’s  goals.  It  follows  from  this 
assumption  that  in  order  for  automation  to  do  its  part  of  the  job,  it  must  communicate  with  the  user. 

Second,  machines  are  literal-minded.  We  have  often  yelled  at  our  machines  to  “Do  what  I  mean,  not  what 
I  said!!!”  but  they  can  only  do  what  we  tell  them  to  do.  Humans  are  good  at  understanding  intention  and 
context.  Machines  are  not.  For  example,  the  machine  -  world  interface  is  limited  to  what  a  pre-determined 
subset  of  possible  inputs.  The  automation  might  have  privileged  access  to  information  that  the  human 
cannot  access  alone.  While  machines  may  have  privileged  access,  they  can  only  interpret  those  inputs  in  a 
pre-planned  way.  Surprise  can  not  easily  be  accommodated. 

Third,  users  have  specific  expectations  about  how  machines  will  work  and  respond  accordingly.  We  expect 
machines  to  work  consistently,  the  same  way  every  time  the  situation  is  the  same.  When  automation  works  in 
unpredictable  ways  humans  do  know  how  to  interpret  the  behavior  and  may  not  trust  the  information. 
We  expect  other  people  to  differ  in  their  behavior  from  time  to  time,  but  not  automation.  We  might  discount 
the  poor  performance  of  a  co-worker  who  has  a  bad  cold.  Sniffling  and  sneezing  are  cues  to  the  condition. 
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Another  day,  the  same  eo-worker  might  be  espeeially  aeeommodating  or  faster  to  answer  a  question.  We  ean 
adapt  to  these  intra-person  fluetuations  by  observing  the  interpersonal  signs,  ineluding  direct  communication. 
What  are  the  cues  that  a  machine  is  under  (or  over)  performing?  When  automation  behaves  in  an  unexpected 
way,  we  tend  to  attribute  it  to  a  breakdown  in  functioning  (hardware  or  software)  and  reject  all  subsequent 
output.  As  we  move  to  adaptive  automation,  great  care  must  be  taken  to  provide  cues  as  to  the  state  of  the 
system  so  that  user  and  automation  can  continue  to  function  as  a  team. 


AUTOMATION 

Increasing  Automation  (Both  Military  and  Non-Military) 

Automation  increases  our  reach,  thrust,  memory,  computational  power,  and  speed.  It  has  always  been  a  part  of 
work  and  of  warfare.  For  example,  the  bow  increased  the  range  that  a  hunter  or  warrior  could  project  his 
reach.  With  more  automation  even  greater  range  is  possible  and  the  desire  for  increased  range  promotes 
greater  and  greater  automation.  However,  the  archer  could  usually  see  his  target.  The  down  side  of  greater 
range  is  that  the  throw  is  greater  than  the  warrior’s  ability  to  see  the  target.  Again,  technology  comes  to  the 
rescue  with  remote  sensors.  The  trade-off  is  increased  uncertainty  about  the  target  identification  and  location. 
Thus,  automation  is  a  player  in  both  creating  and  mitigating  the  effects  of  the  “fog  of  war.” 

Automation  and  Uncertainty 

Automated  systems  now  do  everything  from  fly  our  airplanes  to  predict  the  weather.  They  use  algorithms, 
statistical  models,  artificial  intelligence,  and  other  kinds  of  programs  to  perform  their  work.  Automation  does 
not  work  alone.  Even  at  its  most  autonomous,  it  furthers  the  goals  of  humans  who  must  directly  or  indirectly 
interact  with  the  automation.  Thus,  there  is  (almost)  always  a  person  working  with  the  automated  systems. 
The  question  is,  how  much  does  the  person  know  about  the  automation  and  how  certain  is  the  information 
received  from  the  system?  The  answers  to  these  questions  are  at  the  heart  of  this  paper. 


UNCERTAINTY 

Defined 

Uncertainty  occurs  in  any  situation  in  which  the  decision  maker  does  have  perfect  knowledge  of  the  true  state 
of  the  world  and  /  or  the  effects  of  contemplated  actions.  Causes  include: 

1 .  Observation  uncertainty  such  as: 

signal  /  sensor  error 
transmission  loss  or  distortion 
underrepresented  information 
data  age. 

2.  Processing  uncertainty  such  as: 

statistical  estimation 

aggregation 

smoothing 

misapplication  of  assumptions 

error  propagation  from  previous  calculations 

rule  conflicts  (AI  methods) 
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3.  Reporting  uncertainty  such  as: 

misrepresentation  of  uncertainty 
failure  to  communicate  with  the  user 
poor  format  for  displaying  information. 

Uncertainty  and  Automation 

While,  in  one  sense,  uncertainty  is  in  the  head  of  the  user,  it  is  also  a  feature  of  information.  Colloquially, 
we  speak  of  the  information  being  uncertain  whenever  it  is  unreliable,  for  any  reason.  Automation  can  mask 
uncertainty  due  to  noise,  unreliable  measurements,  missing  data,  etc.,  can  amplify  uncertainty  with  statistical 
or  AI  models  inappropriately  applied,  and  can  create  uncertainty  in  the  head  of  the  user  when  he  or  she  does 
not  know  what  the  system  is  likely  to  do.  For  example,  weather  models  use  previous  runs  as  inputs  to 
successive  runs.  Thus,  when  a  model  is  wrong,  it  propagates  the  errors.  Despite  this  propagation,  there  is  no 
indication  of  uncertainty  in  the  model  output,  a  very  complex  graphic.  Figure  1  is  the  automatically  generated 
output  of  a  weather  modeling  /  forecasting  system.  It  is  typical  of  many  automated  systems.  It  presents 
complex  information  in  a  very  complex  visualization  and  it  presents  all  its  information  as  if  it  were  very 
precisely  known.  In  reality,  however,  there  are  very  large  levels  of  uncertainty  about  much  of  the  information 
in  this  display  (exact  size,  orientation,  location,  source,  etc.). 


FNMQC  NQGAPS  20Q20510Q0  run  1,0*  Fields  T 


850mh  Temperatures  RH  /  Winds 


Figure  1 
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Uncertainty  and  Expertise 

Experts  are  very  aware  of  uneertainty.  For  example,  the  expert  submariner  in  Figure  2  is  expressing  both  a 
spatial  understanding  of  conical  angle  and  the  associated  uncertainty  by  gesture  that  includes  directional  hand 
position  and  dynamic  motion  for  the  degree  of  uncertainty.  He  is  saying.  “To  me  that’s  always  a  challenge, 
to  take  the  conical  angle  and  convert  it  into  the  possible  relative  bearings,  you  know,  along  the  cone  and  try  to 
figure  out  what  that  means  to  me.” 


Figure  2:  Submarine  Officer  Uses  Gesture  that  Indicates  Uncertainty. 

Experts  are  aware  that  there  is  uncertainty  in  most  situations.  They  don’t  simply  trust  the  system-generated 
answer.  For  example,  in  an  experiment  with  three  levels  of  expertise,  the  experts  showed  a  pattern  of 
evaluating  the  reliability  of  automated  data  by  checking  it  against  “raw”  data.  Neither  the  intermediate 
journeymen  nor  the  novices  did  this.  Both  were  less  likely  to  look  at  raw  data  and  more  likely  to  believe  the 
automated  information  (see  Figure  3). 
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Computed  data  Measured  data 


Figure  3:  Experts  Use  Virtually  Equal  Amounts  of  Computer-Analyzed  and 
Raw  Data.  Non-experts  use  significantly  more  computer-analyzed  data. 


In  a  third  example,  weather  models  system  that  provide  no  uneertainty  information  (see  Figure  1).  To  evaluate 
how  aeeurate  the  models  are,  weather  forecasters  comparing  models  against  one  another,  against  current 
conditions,  and  over  time  (see  Figure  4).  In  this  way  they  can  evaluate  the  incalculable,  hidden  uncertainty. 


Figure  4:  Weather  Forecaster  Comparing  Predictions  from  Two  Models. 
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The  Representation  Match  Hypothesis 

Knowing  that  experts  seek  to  evaluate  the  uneertainty  of  automation-provided  information  does  not  provide 
guidanee  on  how  to  display  uneertainty.  The  representation  match  hypothesis  provides  one  answer.  It  states 
that  processing  of  uncertainty  will  be  most  effective  (least  errors,  least  time)  when  the  representations  of 
uncertainty  match  the  representations  required  for  problem  solving.  When  there  is  a  mismatch,  additional 
errors  or  processing  will  occur.  We  propose  three  hypotheses  that  examine  this  issue  at  three  different  levels. 
The  first  variant  hypothesizes  that  the  match  of  external  to  internal  representations  is  an  important  aspect  of 
visualization  uncertainty  understanding.  The  more  cognitive  work  a  person  has  to  go  through  to  match  internal 
to  external  representation  (or  vice  versa),  the  more  difficult  the  task  will  be. 

Some  Data 

For  example,  traditionally,  Target-Motion  Analysis  (TMA)  uncertainty  was  displayed  as  a  verbal  descriptor. 
When  the  engineers  computed  a  robust  numerical  estimate  of  the  uncertainty  around  1990,  they  proposed 
displaying  it  as  an  Area  of  Uncertainty  (AOU)  around  the  contact  position.  Kirschenbaum  and  Arruda 
[1994  #991]  then  tested  the  old  verbal  uncertainty  against  the  new  spatial  display.  Sixteen  fleet  experts  were 
given  simulated  scenarios  with  two  levels  of  difficulty  (as  a  function  of  ocean  noise).  There  were  two  different 
displays  of  (spatial)  uncertainty:  one  verbal  (i.e.,  different  format)  and  one  spatial  (i.e.,  same  format). 
Although  there  were  no  differences  in  speed  across  condition,  there  were  significant  differences  in  the  range 
error  such  that  the  errors  in  the  verbal  condition  were  significantly  higher  than  in  the  spatial  condition, 
especially  for  high  difficulty  tasks  (see  Figure  5). 


Figure  5:  Range  Error  Noise  with  Verbai  and  Graphicai 
Dispiay  of  Uncertainty  as  a  Function  of  Ocean  Noise. 

In  ongoing  research  we  are  employing  an  eye-tracker  to  follow  the  user’s  focus  of  attention  in  a  highly 
uncertain  submarine  problem.  This  problem  employs  multiple  automated  algorithms  and  displays  of 
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associated  uncertainty.  Figure  6  shows  10  fixations  of  sample  data  for  one  user  from  early  in  one  scenario. 
Again,  the  expert  user  is  assessing  the  uncertainty  in  the  automation  by  comparing  different  answers  and  the 
displayed  uncertainties. 
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Figure  6:  Ten  Fixations  of  Sample  Data  for  One  User  from  Early  in  the 
Scenario.  Each  window  displays  uncertainty  in  a  different  format. 


CONCLUSION 

The  two  most  important  conclusions  that  can  be  drawn  from  this  paper  are  that  uncertainty  matters  and  that 
communication  between  system  and  user  is  necessary,  both  to  support  user  confidence  and  to  reduce  errors. 
Uncertainty  is  inherent  in  military  operations  and  in  the  non-military  world,  yet  it  is  often  ignored.  By  making 
assumptions,  explicitly  or  implicitly,  about  the  environment  in  which  it  works,  automation  can  modify 
evidence  of  uncertainty,  conceal  the  sources  of  variability,  or  even  induce  uncertainty  in  the  user  by 
neglecting  environmental  variability.  By  masking  that  uncertainty,  automation  can  induce  human  errors  or 
delay  decision  making  while  the  expert  searches  for  confirming  evidence.  The  Representation  Match 
Hypothesis  can  guide  the  display  of  uncertainty  to  alleviate  these  problems.  Finally,  automation  must 
collaborate  with  the  user  to  improve  the  ability  for  both  to  understand  the  situation. 
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